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[Name of Document] CLAIMS 
[Claim 1] 

An information processing apparatus comprising: 

encoding means for encoding an input stream so as to 
include, among a base stream and first to n-th extension 
streams having extensibility for the base stream, at least 
the base stream and the first extension stream; 

table generating means for generating a table in which 
information is written for associating IDs that respectively 
identify the base stream and the first to n-th extension 
streams, which are encoded by the encoding means, with the 
base stream and the first to n-th extension streams; 

adding means for adding the corresponding IDs to the 
base stream and the first to n-th extension streams encoded 
by the encoding means; and 

packetizing means for packetizing the base stream and 
the first to n-th extension streams, to which the IDs are 
added by the adding means, and the table into TS packets. 
[Claim 2] 

An information processing method comprising: 
an encoding step of encoding an input stream so as to 
include, among a base stream and first to n-th extension 
streams having extensibility for the base stream, at least 
the base stream and the first extension stream; 

a table generating step of generating a table in which 
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information is written for associating IDs that respectively 
identify the base stream and the first to n-th extension 
streams, which are encoded by processing in the encoding 
step, with the base stream and the first to n-th extension 
streams ; 

an adding step of adding the corresponding IDs to the 
base stream and the first to n-th extension streams encoded 
by processing in the encoding step; and 

a packetizing step of packetizing the base stream and 
the first to n-th extension streams, to which the IDs are 
added by processing in the adding step, and the table into 
TS packets. 
[Claim 3] 

A program for allowing a computer to execute processing 
including : 

an encoding step of encoding an input stream so as to 
include, among a base stream and first to n-th extension 
streams having extensibility for the base stream, at least 
the base stream and the first extension stream; 

a table generating step of generating a table in which 
information is written for associating IDs that respectively 
identify the base stream and the first to n-th extension 
streams, which are encoded by processing in the encoding 
step, with the base stream and the first to n-th extension 
streams ; 
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an adding step of adding the corresponding IDs to the 
base stream and the first to n-th extension streams encoded 
by processing in the encoding step; and 

a packetizing step of packetizing the base stream and 
the first to n-th extension streams, to which the IDs are 
added by processing in the adding step, and the table into 
TS packets. 
[Claim 4] 

An information processing apparatus comprising: 
input means for inputting a stream including TS packets 
forming a base stream, TS packets forming each of first to 
n-th extension streams having extensibility for the base 
stream, and a TS packet storing a table in which information 
is written for associating IDs that respectively identify 
the TS packets with the base stream or the first to n-th 
extension streams formed of the TS packets; 

determining means for referring to the table stored in 
the TS packet input by the input means and determining the 
type of processable stream; 

selecting means for selecting, from the stream, the TS 
packets having the ID associated with the stream determined 
by the determining means to be processable; and 

decoding means for decoding the TS packets selected by 
the selecting means. 
[Claim 5] 
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The information processing apparatus according to Claim 
4, further comprising: 

buffering means for buffering, with respect to each ID, 
the TS packets selected by the selecting means. 
[Claim 6] 

An information processing method comprising: 

an input step of inputting a stream including TS 
packets forming a base stream, TS packets forming each of 
first to n-th extension streams having extensibility for the 
base stream, and a TS packet storing a table in which 
information is written for associating IDs that respectively 
identify the TS packets with the base stream or the first to 
n-th extension streams formed of the TS packets; 

a determining step of referring to the table stored in 
the TS packet input by processing in the input step and 
determining the type of processable stream; 

a selecting step of selecting, from the stream, the TS 
packets having the ID associated with the stream determined 
by processing in the determining step to be processable; and 

a decoding step of decoding the TS packets selected by 
processing in the selecting step. 
[Claim 7] 

A program for allowing a computer to execute processing 
including : 

an input step of inputting a stream including TS 
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packets forming a base stream, TS packets forming each of 
first to n-th extension streams having extensibility for the 
base stream, and a TS packet storing a table in which 
information is written for associating IDs that respectively 
identify the TS packets with the base stream or the first to 
n-th extension streams formed of the TS packets; 

a determining step of referring to the table stored in 
the TS packet input by processing in the input step and 
determining the type of processable stream; 

a selecting step of selecting, from the stream, the TS 
packets having the ID associated with the stream determined 
by processing in the determining step to be processable; and 

a decoding step of decoding the TS packets selected by 
processing in the selecting step. 
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[Name of Document] SPECIFICATION 

[Title of the Invention] INFORMATION PROCESSING APPARATUS, 
INFORMATION PROCESSING METHOD, AND PROGRAM 

[Technical Field] 
[0001] 

The present invention relates to information processing 
apparatuses, information processing methods, and programs, 
and in particular, to an information processing apparatus, 
an information processing method, and a program, for 
processing a stream including a base stream and extension 
streams at a plurality of levels. 
[Background Art] 
[0002] 

An MPEG (Moving Picture Experts Group) 2 audio stream 
has backward compatibility so that it can be played back by 
even an MPEG1 audio decoder. In other words, the MPEG2 
audio stream has a structure including an MPEGl audio stream 
portion as a base portion and an MPEG2 audio portion as an 
extension portion thereof. 
[0003] 

In the DVD (Digital Versatile Disc) video format, a 
technology for multiplexing an MPEG2 audio stream into a 
program stream is disclosed (e.g., Non-Patent Document 1). 
Fig. 1 is an illustration of the structure of a program 
stream 1 in the DVD video format. The program stream 1 in 
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Fig. 1 includes a video pack 11, an MPEG2 audio pack 12, and 
a plurality of packs 13-1 to 13-j (j represents an arbitrary 
natural number) . 
[0004] 

The MPEG2 audio pack 12 includes a pack header 21, a 
PES (Packetized Elementary Stream) packet header 22, an 
MPEG1 audio data (Base) 23, a PES packet header 24, and an 
MPEG2 audio data (Extension) 25. In addition, the payload 
of the MPEG2 audio pack 12 includes an MPEG1 audio PES 
packet including the PES packet header 22 and the MPEG1 
audio data 23, and an MPEG2 audio extension PES packet 
including the PES packet header 24 and the MPEG2 audio data 
25. 

[0005] 

When playing back the MPEG2 audio pack 12, a playback 
apparatus that can decode only an MPEG1 audio stream 
(playback apparatus only for MPEG1) separates and plays back 
only the PES packet header 22 and the MPEG1 audio data 23, 
which correspond to the MPEG1 audio stream portion. A 
playback apparatus that can perform decoding of up to an 
MPEG2 audio stream (playback apparatus capable of playing 
back up to an extension audio stream) separates and plays 
back both base and extension audio streams. Specifically, 
the latter playback apparatus plays back, in addition to the 
PES packet header 22 and the MPEG1 audio data 23, the PES 
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packet header 24 and the MPEG2 audio data 25, which 
correspond to the MPEG2 extension audio stream. 

[Non-Patent Document 1] DVD Specifications for Read- 
only Disc Part 3; Versionl.l 
[Disclosure of the Invention] 

[Problems to be Solved by the Invention] 
[0006] 

However, for stream extensibility in the recent years, 
in the case of the program stream shown in Fig. 1, the 
stream structure is determined as a format. Thus, there is 
a problem in that a further audio stream extension cannot be 
added. 

[0007] 

This results in reduction of extensibility and causes a 
problem in that it is difficult to realize standardization 
of playback apparatuses. 

[0008] 

In addition, when an audio stream is broadcast, it is 
common to perform broadcasting by using a TS (Transport 
Stream) . However, the use of a TS is inappropriate in the 
program stream encoding method shown in Fig. 1. 
Specifically, a TS packet corresponding to a pack (e.g., the 
MPEG2 audio pack 12 in Fig. 1) has a relatively small length 
of 188 bytes. When two PES packets, a base portion (MPEG1) 
and an extension portion (MPEG2), are multiplexed in the TS 
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packet, there is a problem in that the encoding efficiency 
is low. 

[0009] 

The present invention has been made in view of the 
above circumstances and is intended to enable encoding and 
decoding of a stream including a base stream and extension 
streams at a plurality of levels. 
[Means for Solving the Problems] 
[0010] 

A first information processing apparatus of the present 
invention includes encoding means for encoding an input 
stream so as to include, among a base stream and first to n- 
th extension streams having extensibility for the base 
stream, at least the base stream and the first extension 
stream; table generating means for generating a table in 
which information is written for associating IDs that 
respectively identify the base stream and the first to n-th 
extension streams, which are encoded by the encoding means, 
with the base stream and the first to n-th extension 
streams; adding means for adding the corresponding IDs to 
the base stream and the first to n-th extension streams 
encoded by the encoding means; and packetizing means for 
packetizing the base stream and the first to n-th extension 
streams, to which the IDs are added by the adding means, and 
the table into TS packets. 
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[0011] 

A first information processing method of the present 
invention includes an encoding step of encoding an input 
stream so as to include, among a base stream and first to n- 
th extension streams having extensibility for the base 
stream, at least the base stream and the first extension 
stream; a table generating step of generating a table in 
which information is written for associating IDs that 
respectively identify the base stream and the first to n-th 
extension streams, which are encoded by processing in the 
encoding step, with the base stream and the first to n-th 
extension streams; an adding step of adding the 
corresponding IDs to the base stream and the first to n-th 
extension streams encoded by processing in the encoding 
step; and a packetizing step of packetizing the base stream 
and the first to n-th extension streams, to which the IDs 
are added by processing in the adding step, and the table 
into TS packets. 
[0012] 

A first program of the present invention is a program 
for allowing a computer to execute processing including an 
encoding step of encoding an input stream so as to include, 
among a base stream and first to n-th extension streams 
having extensibility for the base stream, at least the base 
stream and the first extension stream; a table generating 
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step of generating a table in which information is written 
for associating IDs that respectively identify the base 
stream and the first to n-th extension streams, which are 
encoded by processing in the encoding step, with the base 
stream and the first to n-th extension streams; an adding 
step of adding the corresponding IDs to the base stream and 
the first to n-th extension streams encoded by processing in 
the encoding step; and a packetizing step of packetizing the 
base stream and the first to n-th extension streams, to 
which the IDs are added by processing in the adding step, 
and the table into TS packets. 
[0013] 

A second information processing apparatus of the 
present invention includes input means for inputting a 
stream including TS packets forming a base stream, TS 
packets forming each of first to n-th extension streams 
having extensibility for the base stream, and a TS packet 
storing a table in which information is written for 
associating IDs that respectively identify the TS packets 
with the base stream or the first to n-th extension streams 
formed of the TS packets; determining means for referring to 
the table stored in the TS packet input by the input means 
and determining the type of processable stream; selecting 
means for selecting, from the stream, the TS packets having 
the ID associated with the stream determined by the 
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determining means to be processable; and decoding means for 
decoding the TS packets selected by the selecting means. 
[0014] 

The second information processing apparatus may further 
include buffering means for buffering, with respect to each 
ID, the TS packets selected by the selecting means. 
[0015] 

A second information processing method of the present 
invention includes an input step of inputting a stream 
including TS packets forming a base stream, TS packets 
forming each of first to n-th extension streams having 
extensibility for the base stream, and a TS packet storing a 
table in which information is written for associating IDs 
that respectively identify the TS packets with the base 
stream or the first to n-th extension streams formed of the 
TS packets; a determining step of referring to the table 
stored in the TS packet input by processing in the input 
step and determining the type of processable stream; a 
selecting step of selecting, from the stream, the TS packets 
having the ID associated with the stream determined by 
processing in the determining step to be processable; and a 
decoding step of decoding the TS packets selected by 
processing in the selecting step. 
[0016] 

A second program of the present invention is a program 
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for allowing a computer to execute processing including an 
input step of inputting a stream including TS packets 
forming a base stream, TS packets forming each of first to 
n-th extension streams having extensibility for the base 
stream, and a TS packet storing a table in which information 
is written for associating IDs that respectively identify 
the TS packets with the base stream or the first to n-th 
extension streams formed of the TS packets; a determining 
step of referring to the table stored in the TS packet input 
by processing in the input step and determining the type of 
processable stream; a selecting step of selecting, from the 
stream, the TS packets having the ID associated with the 
stream determined by processing in the determining step to 
be processable; and a decoding step of decoding the TS 
packets selected by processing in the selecting step. 
[0017] 

According to the first present invention, an input 
stream is encoded so as to include, among a base stream and 
first to n-th extension streams, at least the base stream 
and the first extension stream. A table is generated in 
which information is written for associating IDs that 
respectively identify the encoded base stream and the 
encoded first to n-th extension streams with the base stream 
and the first to n-th extension streams. The corresponding 
IDs are added to the encoded base stream and the encoded 
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first to n-th extension streams. The base stream, the first 
to n-th extension streams, and the table are packetized into 
TS packets. 
[0018] 

According to the second present invention, a stream is 
input, which includes TS packets forming a base stream, TS 
packets forming each of first to n-th extension streams, and 
a TS packet storing a table in which information is written 
for associating IDs that respectively identify the TS 
packets with the base stream or the first to n-th extension 
streams formed of the TS packets. The table is referred to 
and the type of processable stream is determined. From the 
stream, the TS packets having the ID associated with the 
stream determined to be processable are selected and decoded. 
[Advantages] 
[0019] 

According to the first present invention, processing in 
accordance with stream extension can be performed. In 
particular, according to the present invention, even when a 
stream is extended, encoding in accordance with an 
information processing apparatus at a receiving side can be 
performed. 

[0020] 

According to the second present invention, processing 
in accordance with stream extension can be performed. In 
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particular, according to the present invention, even when an 
extended stream is input, decoding in accordance with the 
processing capacity of the information processing apparatus 
can be performed. 

[Best Mode for Carrying Out the Invention] 
[0021] 

Embodiments of the present invention will be described 
below. The following description indicates correspondence 
relationships between the invention described in this 
specification and the embodiments of the invention as 
examples. This description is used to confirm that the 
embodiments described in this specification and supporting 
the invention are described in this specification. 
Therefore, even if there is an embodiment not here described 
as a correspondence to the invention but described in the 
embodiments of the invention, it does not mean that the 
embodiment does not correspond to the invention. In 
contrast, even if an embodiment is here described as a 
correspondence of the invention, it does not mean that the 
embodiment does not correspond to inventions other than that 
invention . 

[0022] 

Further, this description does not mean all the 
inventions described in this specification. In other words, 
this description does not deny existence of inventions 
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described in this specification and not claimed in this 
application, namely, existence of inventions that will be 
divisionally applied, will be appeared by amendment, and 
will be added in future. 
[0023] 

The information processing apparatus (for example, the 
transmitter 41 in Fig. 3) described in claim 1 includes 
encoding means (for example, the audio encoder 72 in Fig. 3 
for executing the process of step S13 in Fig. 6) for 
encoding an input stream (for example, the audio stream) so 
as to include, among a base stream and first to n-th 
extension streams having extensibility for the base stream, 
at least the base stream and the first extension stream; 
table generating means (for example, the extraction- 
information adding section 75 in Fig. 3 for executing the 
process of step S17 in Fig. 6) for generating a table (for 
example, the table in which PAT and PMT are written in Fig. 
7 and Fig. 8) in which information is written for 
associating IDs (for example, PID) that respectively 
identify the base stream and the first to n-th extension 
streams, which are encoded by the encoding means, with the 
base stream and the first to n-th extension streams; adding 
means (for example, the TS packetizing section 76 in Fig. 3 
for executing the process of step S33 in Fig. 9) for adding 
the corresponding IDs to the base stream and the first to n- 
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th extension streams encoded by the encoding means; and 
packetizing means (for example, the TS packetizing section 
76 3 for executing the processes of step S32 and step S33 in 
Fig. 9) for packetizing the base stream and the first to n- 
th extension streams, to which the IDs are added by the 
adding means, and the table into TS packets. 
[0024] 

The information processing method described in claim 2 
includes an encoding step (for example, step S13 in Fig. 6) 
of encoding an input stream (for example, the audio stream) 
so as to include, among a base stream and first to n-th 
extension streams having extensibility for the base stream, 
at least the base stream and the first extension stream; a 
table generating step (for example, step S17 in Fig. 6) of 
generating a table (for example, the table in which PAT and 
PMT are written in Fig. 7 and Fig. 8) in which information 
is written for associating IDs (for example, PID) that 
respectively identify the base stream and the first to n-th 
extension streams, which are encoded by processing in the 
encoding step, with the base stream and the first to n-th 
extension streams; an adding step (for example, step S33 in 
Fig. 9) of adding the corresponding IDs to the base stream 
and the first to n-th extension streams encoded by 
processing in the encoding step; and a packetizing step (for 
example, step S32 and step S34 in Fig. 9) of packetizing the 
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base stream and the first to n-th extension streams, to 
which the IDs are added by processing in the adding step, 
and the table into TS packets. 
[0025] 

The program described in claim 3 for allowing a 
computer to execute processing including an encoding step 
(for example, step S13 in Fig. 6) of encoding an input 
stream (for example, the audio stream) so as to include, 
among a base stream and first to n-th extension streams 
having extensibility for the base stream, at least the base 
stream and the first extension stream; a table generating 
step (for example, step S17 in Fig. 6) of generating a table 
(for example, the table in which PAT and PMT are written in 
Fig. 7 and Fig. 8) in which information is written for 
associating IDs (for example, PID) that respectively 
identify the base stream and the first to n-th extension 
streams, which are encoded by processing in the encoding 
step, with the base stream and the first to n-th extension 
streams; an adding step (for example, step S33 in Fig. 9) of 
adding the corresponding IDs to the base stream and the 
first to n-th extension streams encoded by processing in the 
encoding step; and a packetizing step (for example, step S32 
and step S34 in Fig. 9) of packetizing the base stream and 
the first to n-th extension streams, to which the IDs are 
added by processing in the adding step, and the table into 
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TS packets. 
[0026] 

The information processing apparatus (for example, the 
audio-stream processing section 122 in Fig. 16) described in 
claim 4 includes input means (for example, the input portion 
151 in Fig. 16) for inputting a stream including TS packets 
(for example, the TS packet 92 in Fig. 5) forming a base 
stream, TS packets (for example, the TS packets 93-1 to 93-n 
in Fig. 5) forming each of first to n-th extension streams 
having extensibility for the base stream, and a TS packet 
(for example, the TS packets 90, 91 in Fig. 5) storing a 
table (for example, Fig. 7, Fig. 8) in which information is 
written for associating IDs that respectively identify the 
TS packets with the base stream or the first to n-th 
extension streams formed of the TS packets; determining 
means (for example, the filter control portion 152 in Fig. 
16 for executing the process of step S94 in Fig. 17) for 
referring to the table stored in the TS packet input by the 
input means and determining the type of processable stream; 
selecting means (for example, the PID filter 261 in Fig. 16 
for executing the process of step S95 in Fig. 17) for 
selecting, from the stream, the TS packets having the ID 
associated with the stream determined by the determining 
means to be processable; and decoding means (for example, 
the audio decoder 262 in Fig. 16 for executing the process 
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of step S98 in Fig. 17) for decoding the TS packets selected 
by the selecting means. 
[0027] 

The information processing apparatus described in claim 
5, further includes buffering means (for example, the base 
buffer 154 and the extension buffers 202-1 to 202-n in Fig. 

16) for buffering, with respect to each ID, the TS packets 
selected by the selecting means. 

[0028] 

The information processing method described in claim 6 
includes an input step (for example, step S91 in Fig. 17) of 
inputting a stream including TS packets (for example, the TS 
packet 92 in Fig. 5) forming a base stream, TS packets (for 
example, the TS packets 93-1 to 93-n in Fig. 5) forming each 
of first to n-th extension streams having extensibility for 
the base stream, and a TS packet (for example, the TS 
packets 90, 91 in Fig. 5) storing a table (for example, Fig. 
7, Fig. 8) in which information is written for associating 
IDs that respectively identify the TS packets with the base 
stream or the first to n-th extension streams formed of the 
TS packets; a determining step (for example, step S94 in Fig. 

17) of referring to the table stored in the TS packet input 
by processing in the input step and determining the type of 
processable stream; a selecting step (for example, step 
S95 in Fig. 17) of selecting, from the stream, the TS 
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packets having the ID associated with the stream determined 
by processing in the determining step to be processable; and 
a decoding step (for example, step S98 in Fig. 17) of 
decoding the TS packets selected by processing in the 
selecting step. 
[0029] 

The program described in claim 7 for allowing a 
computer to execute processing including: an input step (for 
example, step S91 in Fig. 17) of inputting a stream 
including TS packets (for example, the TS packet 92 in Fig. 
5) forming a base stream, TS packets (for example, the TS 
packets 93-1 to 93-n in Fig. 5) forming each of first to n- 
th extension streams having extensibility for the base 
stream, and a TS packet (for example, the TS packets 90, 91 
in Fig. 5) storing a table (for example, Fig. 7, Fig. 8) in 
which information is written for associating IDs that 
respectively identify the TS packets with the base stream or 
the first to n-th extension streams formed of the TS 
packets; a determining step (for example, step S94 in Fig. 
17) of referring to the table stored in the TS packet input 
by processing in the input step and determining the type of 
processable stream; a selecting step (for example, step S95 
in Fig. 17) of selecting, from the stream, the TS packets 
having the ID associated with the stream determined by 
processing in the determining step to be processable; and a 
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decoding step (for example, step S98 in Fig. 17) of decoding 
the TS packets selected by processing in the selecting step. 
[0030] 

Embodiments of the present invention will be described 
below with reference to drawings. 
[0031] 

Fig. 2 is a diagram showing an example of the 
configuration of the entirety of a transmitting/receiving 
system to which the present invention is applied. 

[0032] 

This transmitting/receiving system 40 includes a 
transmitter 41 and a receiver 42. The transmitter 41 is an 
apparatus, such as a broadcasting station, for transmitting 
a stream including a plurality of TS packets. The receiver 
42 is a household apparatus, such as a household set-top box, 
for receiving the stream. 

[0033] 

In this embodiment, the transmitter 41 encodes an audio 
stream to generate TS packets and transmits the TS packets, 
and the receiver 42 receives the TS packets and decodes the 
TS packets to acquire the audio stream. 

[0034] 

Fig. 3 is a block diagram showing an example of the 
configuration of the transmitter 41 in Fig. 2. 
[0035] 
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The transmitter 41 includes an input section 71, an 
audio encoder 72, a base buffer 73, extension buffers 74-1 
to 74-n, an extraction-information adding section 75, a TS 
packetizing section 76, and a transmitting section 77, in 
which n represents an arbitrary natural number equal to one 
or greater. 
[0036] 

An audio stream to be transmitted is input to the input 
section 71. The audio encoder 72 encodes the audio stream. 
The audio encoder 72 in Fig. 2 is an encoder corresponding 
to the n-th extension audio stream. That is, the audio 
encoder 72 can encode the audio stream into a base stream 
and extension audio streams at a plurality of levels ranging 
from the first to n-th levels. 
[0037] 

In this embodiment, the larger the value of the level n, 
the higher the extensibility is, resulting in better audio 
playback quality and higher functionality. The audio 
encoder 72 encodes the audio stream into the base stream and 
the first to n-th extension streams and supplies the encoded 
streams to the corresponding base buffer 73 the extension 
buffers 74-1 to 74-n at the corresponding levels. For 
example, the audio encoder 72 supplies the encoded base 
audio stream (BS) to the base buffer 73, supplies the 
encoded first extension audio stream (Extl) to the extension 
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buffer 74-1, supplies the encoded second extension audio 
stream (Ext2) to the extension buffer 74-2, and similarly 
supplies the encoded n-th extension audio stream (ExtN) to 
the extension buffer 74-n. Here, the level of the extension 
audio stream corresponds to the reference numeral of the 
buffer. Also, the base audio stream is represented as BS, 
and the first to n-th extension audio streams are 
represented as Extl to ExtN, respectively. 
[0038] 

After separating the audio stream into the base stream 
and the first to n-th extension audio streams, the audio 
encoder 72 in this embodiment may encode the streams. 
Alternatively, as the result of encoding the audio stream, 
the base stream and the first to n-th extension audio 
streams may be output. 
[0039] 

The base buffer 73 stores (buffers) the base audio 
stream, and the extension buffers 74-1 to 74-n store 
(buffer) the first to n-th extension audio streams, 
respectively. Under the control of the TS packetizing 
section 76, the base buffer 73 and the extension buffers 74- 
1 to 74-n read the audio streams stored therein. 
[0040] 

The extraction-information adding section 75 generates 
tables, which are extraction information, so that at the 
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decoding side the extension audio stream at a desired level 
is extracted from the base audio stream and the first to n- 
th extension audio streams. In the tables, information is 
written for associating IDs (PIDs (Packet Identification)) 
that respectively identify the base audio stream and the 
first to n-th extension audio streams with the base audio 
stream and the first to n-th extension audio streams. 
Specifically, the tables include a PAT (Program Association 
Table) and a PMT (Program Map Table) . The details of the 
tables are described below with reference to Figs. 7 and 8. 
The extraction-information adding section 75 supplies the 
tables to the TS packetizing section 76. 
[0041] 

The TS packetizing section 76 controls the base buffer 
73 and the extension buffers 74-1 to 74-n to acquire the 
base audio stream and the first to n-th extension audio 
streams and also acquires the tables supplied from the 
extraction-information adding section 75. Also, the TS 
packetizing section 76 packetizes the tables into TS packets 
and, based on the tables, packetizes the base audio stream 
and the first to n-th extension audio streams into TS 
packets (generates TS packets) . At this time, the TS 
packetizing section 76 adds the PIDs that identify the types 
of streams (the base audio stream and the first to n-th 
extension audio streams) based on the tables. The TS 
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packetizing section 76 supplies the generated TS packets to 
the transmitting section 77. The transmitting section 77 
transmits the TS packets. Here, the plural TS packets are 
transmitted in sequence, resulting in transmission of one 
stream (consisting of a plurality of TS packets). 
[0042] 

The PID identifies each packet (TS packet (transport 
packet) forming an MPEG TS, and the PID having a unique 
value is added to each packet. In other words, in order 
that the receiver 42 at the receiving side selects a packet 
of an extension audio stream at a desired level, the PID 
value added to this packet is necessary. 

[0043] 

Next, the structure of a TS (transport stream) in which 
the base audio stream and the extension audio streams at a 
plurality of levels are multiplexed is described with 
reference to Figs. 4 and 5. 
[0044] 

In the example of Fig. 4, a TS includes a base audio 
stream 81 and first to n-th extension audio streams 82-1 to 
82-n. Each of the base audio stream 81 and the first to n- 
th extension audio streams 82-1 to 82-n is encoded in units 
of a predetermined number of audio samples , and each unit is 
indicated by a parenthesized subscript. Specifically, for 
example, the base audio stream 81 is separated into a 
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plurality of units BS(1), BS(2), . .., BS (n) and encoded. A 
set of units having the same subscript, for example, BS(1), 
Extl(l), Ext2(l), . .., ExtN(l), is synchronously encoded by 
the audio encoder 72 and synchronously played back (decoded) . 
[0045] 

The TS packetizing section 76 multiplexes the base 
audio stream 81 and the first to n-th extension audio 
streams 82-1 to 82-n into TS packets having different PIDs 
(packet IDs), as shown in Fig. 5. One TS packet stores data 
of, for example, 188 bytes. 
[0046] 

The TS stream in Fig. 5 includes a TS packet 90 in 
which a PAT (Program Association Table) table is written, a 
TS packet 91 in which a PMT (Program Map Table) table is 
written, a TS packet 92 of the base audio stream (BS) with 
PID = aO, a TS packet 93-1 of the first extension audio 
stream (Extl) with PID = al, a TS packet 93-2 of the second 
extension audio stream (Ext2) with PID - a2, and a TS 

packet 93-n of the n-th extension audio stream (ExtN) with 
PID = aN. The TS packet 91 of the table is transmitted by 
the transmitter 41 every predetermined period. 

[0047] 

In this embodiment, the receiver 42 at the playback 
side is capable of decoding at least the base audio stream 
(BS) . When the receiver 42 at the playback side can play 



- 22 - 



back extension audio streams up to a predetermined m-th 
level (m represents a natural number equal to one or greater, 
and m < n) , the receiver 42 can decode the base audio stream 
and the first to m-th extension audio streams. Regarding 
the relationships between the base audio stream and the 
extension audio streams, for example, the larger the value n 
of the decodable extension audio stream, the better the 
audio playback quality is and the higher the functionality 
is . 

[0048] 

Next, a TS packet transmitting process executed by the 
transmitter 41 in Fig. 3 is described with reference to the 
flowchart in Fig. 6. This process is initiated when the 
power of the transmitter 41 is turned on to input an audio 
stream to the input section 71. 

[0049] 

In step Sll, the input section 71 receives the input 
audio stream. In step S12, the input section 71 outputs the 
received audio stream to the audio encoder 72. 

[0050] 

In step S13, the audio encoder 72 encodes the base 
audio stream and the first to n-th extension audio streams. 
As a result, the audio encoder 72 outputs the base audio 
stream and the first to n-th extension audio streams shown 
in Fig . 4 in (vertically) synchronized form. 
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[0051] 

In step S14, the audio encoder 72 outputs the encoded 
audio streams separately for each level (type of stream) . 
Specifically, the audio encoder 72 outputs the encoded base 
audio stream to the base buffer 73, the encoded first 
extension audio stream to the extension buffer 74-1, the 
encoded second extension audio stream to the extension 
buffer 74-2, and the encoded n-th extension audio stream to 
the extension buffer 74-n. 
[0052] 

In step S15, the base buffer 73 and the first to n-th 
extension buffers 74-1 to 74-n respectively store (buffer) 
the encoded audio streams . 

[0053] 

In step S16, the base buffer 73 and the first to n-th 
extension buffers 74-1 to 74-n respectively output the 
encoded audio streams with predetermined timing. Actually, 
the TS packetizing section 76 controls the buffers (the base 
buffer 73 and the first to n-th extension buffers 74-1 to 
74-n) so as to read the corresponding audio streams 
therefrom. 

[0054] 

In step S17, the extraction-information adding section 
75 generates tables and supplies the tables to the TS 
packetizing section 76. Specifically, the extraction- 
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information adding section 75 generates the tables shown in 
Figs. 7 and 8 and supplies the tables to the TS packetizing 
section 76. 
[0055] 

Fig. 7 illustrates the PAT (Program Association Table) . 
Specifically, the PMT-PID corresponding to each program- 
number is written. In the example of Fig. 7, PMT-PID for 
program-number 1 is "X" and PMT-PID for program-number 2 is 
"Y". The value of the PMT-PID is referred to by the PMT 

(Program Map Table) shown in Fig. 8. In Fig. 8, 
stream_entry in the case where the PID is "X" is written. 
Specifically, the PMT includes BASE_PID = aO, Extl_PID = al, 
Ext2_PID = a2, ExtN_PID = aN . It is clear that, 

according to Figs. 7 and 8, the PID of the base audio stream 

(BS) is "aO"; the PID of the first extension audio stream 
Extl is "al"; the PID of the second extension audio stream 
Ext2 is "a2"; and similarly the PID of the n-th extension 
audio stream ExtN is "aN" . Accordingly, as shown in Fig. 5, 
the PID corresponding to the encoding level is identifiable. 
[0056] 

In this embodiment, the PAT and the PMT are transmitted 
in different TS packets. In other words, as shown in Fig. 5, 
the TS packet 90 of the table in which the PAT is written 
and the TS packet 91 of the table in which the PMT is 
written are transmitted. 



- 25 - 



[0057] 

Referring back to Fig. 6, in step S18, the TS 
packetizing section 76 executes a TS packet generating 
process. The details of the process are described below 
with reference to Fig. 9. The TS packets generated by the 
TS packetizing section 76 are output to the transmitting 
section 77. 

[0058] 

In step S19, the transmitting section 77 transmits the 
TS packets (an audio stream including the plural TS packets) 
to the receiver 42. Specifically, the stream including the 
TS packets 90 and 91 of the tables, the TS packet 92 of the 
base audio stream, and the TS packets 93-1 to 93-n of the 
first to n-th extension audio streams is transmitted. After 
that, the process is finished. Although the TS packets are 
transmitted to the receiver 42 in this embodiment, the TS 
packets may be recorded in various types of recording media, 
which are not shown. Alternatively, transmission of the TS 
packets to the receiver 42 may be performed indirectly by 
recording the TS packets in a recording medium and then 
supplying the recording medium to the receiver 42. 

[0059] 

Next, the details of the TS packet generating process 
in step S18 of Fig. 6 are described with reference to the 
flowchart of Fig. 9. 
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[0060] 

In step S31, the TS packetizing section 76 acquires the 
tables. The tables are such as those shown in Figs. 7 and 8, 
which are generated in step S17 of Fig. 6. That is, the 
tables include information in which the PIDs that 
respectively identify the base audio stream and the first to 
n-th extension audio streams and information that 
respectively identifies the base audio stream and the first 
to n-th extension audio streams are written . 

[0061] 

In step S32, the TS packetizing section 76 packetizes 
the tables into TS packets (generates TS packets based on 
the tables) and outputs the TS packets to the transmitting 
section 77 (and the transmitting section 77 transmits the TS 
packets in turn to the receiver 42) . Accordingly, the TS 
packet 90 in which the PAT in Fig. 5 is written and the TS 
packet in which the PMT is written are generated and output 
to the transmitting section 77. In this process, the TS 
packets are transmitted only once. Actually, however, the 
TS packets in which the tables are written are transmitted 
every predetermined period of time. Accordingly, even when 
the receiver 42 starts receiving the stream in the middle 
thereof, the receiver 42 can acquire the tables and thus 
reliably decode the stream. 

[0062] 
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In step S33, the TS packetizing section 76 adds, based 
on the tables, the PIDs to the corresponding audio streams 
from the base buffer 73 and the first to n-th extension 
buffers 74-1 to 74-n. Although the tables are packetized 
into TS packets by the processing in step S31 and then 
transmitted, it is assumed here that the tables are held by 
the TS packetizing section 76. Accordingly, PID = aO is 
added to the base audio stream (BS) , PID = al is added to 
the first extension audio stream (Extl), PID = a2 is added 
to the second extension audio stream (Ext2), and similarly 
PID = aN is added to the n-th extension audio stream (ExtN) . 
[0063] 

In step S35, the TS packetizing section 76 generates TS 
packets, respectively, based on the audio streams from the 
base buffer 73 and the first to n-th extension buffers 74-1 
to 74-n. As shown in Fig. 5, the base audio stream and the 
first to n-th extension audio streams are generated as TS 
packets having respective different PIDs (packet IDs) . In 
other words, the PIDs for determining the types of streams 
are added to the TS packets. 

[0064] 

In step S35, the TS packetizing section 76 outputs the 
generated TS packets to the transmitting section 77. After 
that, the process returns to step S18 of Fig. 6. 

[0065] 
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By the processes in Figs. 6 and 9, the transmitter 41 
that can encode the n-th extension audio stream separates 
the encoded data into the base audio stream and the first to 
n-th extension audio streams; adds the PIDs to the audio 
streams based on the tables; packetizes the separated audio 
streams into TS packets and transmits the TS packets; and 
packetizes the tables into TS packets and transmits the TS 
packets. In other words, the audio stream including the 
plural TS packets having the PIDs identifying the types of 
streams and the TS packets of the tables is transmitted to 
the receiver 42. 
[0066] 

The stream transmitted by the transmitter 41 includes 
the TS packets forming the base audio stream, the TS packets 
forming the first to n-th extension audio streams, and the 
TS packets of the tables in which information is written for 
associating the PIDs that respectively identify these TS 
packets with the base audio stream and the first to n-th 
extension audio streams. Since the PIDs that identify the 
types of audio streams are added to the TS packets forming 
the base audio stream and the TS packets forming the first 
to n-th extension audio streams, the receiving side can 
perform decoding in accordance with its processing capacity. 
The receiver 42 at the receiving side is described below. 
[0067] 
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Fig. 10 is a block diagram showing an example of the 
configuration of the receiver 42 in Fig. 2. 
[0068] 

The receiver 42 includes a receiving section 121, an 
audio-stream processing section 122, and an output section 
123. The receiving section 121 receives the TS packets, and 
the audio-stream processing section 122 performs processing 
concerning an audio stream. Specifically, the audio-stream 
processing section 122 extracts an audio stream by, for 
example, decoding the received TS packets. The output 
section 123 outputs the audio stream processed by the audio- 
stream processing section 122. 
[0069] 

The audio-stream processing section 122 extracts a 
different audio stream depending on its decoding capability. 
The audio-stream processing section 122 is described below. 

[0070] 

Fig. 11 is a diagram showing an example of the 
configuration of the audio-stream processing section 122 
including an audio decoder that can decode only the base 
audio stream. 
[0071] 

The audio-stream processing section 122 in Fig. 11 
includes an input portion 151, a filter control portion 152, 
a PID filter 153, a base buffer 154, and an audio decoder 
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155. 

[0072] 

The input portion 151 receives input of TS packets of 
an audio stream supplied by the receiving section 121 in Fig. 
10. The input portion 151 supplies, among the TS packets of 
the audio stream, the TS packets of the tables (tables 
illustrated in Figs. 7 and 8) to the filter control portion 
152, and the remaining TS packets (e.g., the TS packets of 
the base audio stream and the TS packets of the first to n- 
th extension audio streams having the PIDs in Fig. 5) to the 
PID filter 153. When, for example, TS packets of a video 
stream and of an audio stream are received by the receiving 
section 121, the input portion 151 acquires only the TS 
packets of the audio stream. 
[0073] 

The filter control portion 152 controls the operation 
of the PID filter 153 based on the acquired tables. 
Specifically, the filter control portion 152 remembers the 
type(s) of stream(s) that it can process, and the filter 
control portion 152 determines the type(s) of stream(s) that 
it can process based on the tables. The filter control 
portion 152 refers in the tables to the PID(s) of the 
stream(s) that it can process and controls the PID filter 
153 so as to select the TS packets having the PID(s) 
associated with the stream(s) that it can process. For 
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example, the filter control portion 152 informs the PID 
filter 153 of the value (s) of the PID(s) associated with the 
type(s) of decodable stream(s) as the value (s) of the PID(s) 
to be selected. 
[0074] 

Under the control of the filter control portion 152, 
the PID filter 153 selects (extracts) the TS packets. 
Specifically, the PID filter 153 selects, based on the 
PID(s) reported by the filter control portion 152, the TS 
packets having the same PID(s) and supplies the TS packets 
to the corresponding buffers. Each buffer stores (buffers) 
the TS packets selected by the PID filter 153. The audio 
decoder 155 acquires the TS packets stored in each buffer 
and decodes these TS packets. 
[0075] 

In the example of Fig. 11, the audio-stream processing 
section 122 has only the audio decoder 155 corresponding to 
the base audio stream and is incapable of decoding the first 
to n-th extension audio streams. In this case, the filter 
control portion 152 remembers that the type of stream that 
it can process is only the base audio stream, and the filter 
control portion 152 refers in the tables to the PID 
corresponding to the base audio stream. In this case, the 
audio-stream processing section 122 determines that BASE_PID 
is the type of stream that it can process and refers in the 
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tables to PID = aO . The filter control portion 152 controls 
the PID filter 153 so as to select the TS packets having the 
PID of the decodable stream. For example, the filter 
control portion 152 informs the PID filter 153 of the ID of 
the decodable stream or the value of the PID to be passed 
through. Based on the PID, namely, PID = aO, reported by 
the filter control portion 152, the PID filter 153 selects 
the TS packets having PID = aO and supplies the TS packets 
to the base buffer 154. When the TS packets of the first to 
n-th extension audio streams are supplied, the PID filter 
153 does not select these packets. In other words, when 
only the TS packets of the base audio stream having PID = aO 
are supplied, the PID filter 153 selects the TS packets and 
supplies the TS packets to the base buffer 154 at a 
subsequent stage. 
[0076] 

The base buffer 154 stores the TS packets of the base 
audio stream, which are selected and supplied by the PID 
filter 153. The base buffer 154 operates to allow 
synchronization of the TS packets at the receiving side and 
synchronization with the audio decoder 155. Although only 
one base buffer 154 is provided in the example of Fig. 11, 
two buffers may be provided in serial to serve as the base 
buffer 154, including a buffer for allowing synchronization 
of the TS packets at the receiving side and a buffer for 
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allowing synchronization with the audio decoder 155. The 
audio decoder 155 decodes the TS packets of the base audio 
stream and outputs the decoded base audio stream. 
[0077] 

Next, a TS packet receiving process executed by the 
receiver 42 in Fig. 10 is described with reference to the 
flowchart of Fig. 12. This process is initiated when an 
instruction to receive the TS packets is given to the 
receiver 42 . 

[0078] 

In step S51, the receiving section 121 of the receiver 
42 receives TS packets (a stream including a plurality of TS 
packets) . These TS packets are, for example, the TS packets 
transmitted by the transmitter 41 in step S19 of Fig. 6. 

[0079] 

In step S52, the receiving section 121 extracts the TS 
packets of the audio stream and supplies the TS packets to 
the audio-stream processing section 122 . For example, when 
the TS packets received by the receiving section 121 include 
TS packets of a video stream, the receiving section 121 
extracts only the TS packets of the audio stream and 
supplies the TS packets to the audio-stream processing 
section 122. 

[0080] 

In step S53, the audio- stream processing section 122 
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performs an audio stream processing process for decoding the 
TS packets of the audio stream (the audio stream including 
the plural TS packets) in accordance with the decoding 
capability of the audio-stream processing section 122. The 
details of the process are described with reference to Fig . 
13. The audio stream processed by the audio-stream 
processing section 122 is supplied to the output section 123. 
[0081] 

In step S54, the output section 123 outputs the decoded 
audio stream to, for example, a speaker, which is not shown. 
After that, the process is finished. 

[0082] 

By the process in Fig. 12, the TS packets are received, 
and the TS packets of the audio stream are audio-stream 
processed (decoded) and output . 

[0083] 

Next, a base audio stream processing process, which is 
an example of step S53 of Fig. 12, is described with 
reference to the flowchart of Fig. 13. This process is a 
process executed by the audio-stream processing section 122 
in Fig. 11. Specifically, this process is a process 
executed by the audio-stream processing section 122 that can 
decode only the base audio stream. 

[0084] 

In step S71, the input portion 151 receives input of 
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the TS packets of the audio stream (the audio stream 
including the plural TS packets) . The TS packets of the 
audio stream correspond to the audio stream including the TS 
packets of the tables, the TS packets of the base audio 
stream, and the TS packets of the first to n-th extension 
audio streams, which are transmitted by the transmitter 41 
in step S19 of Fig. 6 described above. 
[0085] 

In step S72, the input portion 151 supplies the TS 
packets of the tables to the filter control portion 152. 
Specifically, since the TS packets of the audio stream 
include the TS packets of the tables, the TS packets of the 
base audio stream, and the TS packets of the first to n-th 
extension audio streams, the input portion 151 supplies, 
among these TS packets, the TS packets of the tables to the 
filter control portion 152 . 

[0086] 

In step S73, the input portion 151 supplies the TS 
packets to which the PIDs are added to the PID filter 153. 
Specifically, in Fig. 5, the input portion 151 supplies the 
TS packets of the base audio stream and the TS packets of 
the first to n-th extension audio streams to which the PIDs 
are added to the PID filter 153. 

[0087] 

In step S74, the filter control portion 152 refers to 
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the tables and determines the type(s) of stream(s) that the 
audio decoder 155 can process. Specifically, the filter 
control portion 152 remembers that the type of stream that 
it can process is the base audio stream and determines, 
based on the tables, the type of stream that it can process. 
The filter control portion 152 refers in the tables to the 
PID of the stream that it can process and controls the PID 
filter 153 so as to select the TS packets having PID = aO 
associated with the base stream that it can process. 
[0088] 

In step S75, under the control of the filter control 
portion 152 (based on the determination) , the PID filter 153 
selects the corresponding TS packets and supplies the TS 
packets to the base buffer 154. Specifically, under the 
control of the filter control portion 152, the PID filter 
153 selects the TS packets having PID - aO, i.e., the TS 
packets of the base audio stream, and supplies the TS 
packets to the base buffer 154. 

[0089] 

In step S76, the base buffer 154 stores the supplied TS 
packets. The TS packets stored here are the TS packets of 
the base audio stream. 

[0090] 

In step S77, the base buffer 154 outputs the TS packets 
to the audio decoder 155 with predetermined timing. 
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[0091] 

In step S78, the audio decoder 155 decodes the supplied 
TS packets of the base audio stream and, in step S79, 
outputs the decoded audio stream, 

[0092] 

Since the PIDs are added to the TS packets and the 
relationships between the PIDs and the types of TS packets 
(TS packets of the base audio stream and the first to n-th 
extension audio streams) are written in the tables, even the 
audio-stream processing section 122 (the receiver 42) that 
can decode only the base audio stream, such as that shown in 
Fig. 11, can select and decode only the TS packets 
corresponding to the base audio stream. That is, even when 
the audio stream extended to a plurality of levels is 
transmitted, the receiver 42 having the audio-stream 
processing section 122 in Fig. 11 can extract only the base 
audio stream and plays back the base audio stream. 

[0093] 

Next, the audio-stream processing section 122 having an 
audio decoder that can decode the base audio stream and the 
first extension audio stream is described with reference to 
Fig. 14. 

[0094] 

Fig . 14 is a diagram showing an example of the 
configuration of the audio-stream processing section 122 
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including an audio decoder that can decode the base audio 
stream and the first extension audio stream. In this 
diagram, portions corresponding to those in Fig. 11 are 
referred to using the same reference numerals, and 
descriptions thereof are omitted to avoid repetition. 
[0095] 

The audio- st ream processing section 12 2 in Fig . 14 
includes, in addition to the audio-stream processing section 
122 in Fig. 11, a first extension buffer 202. An audio 
decoder 203 can decode not only the base audio stream but 
also the first extension audio stream. The PID filter 153 
can extract not only the base audio stream but also the 
first extension audio stream. 

[0096] 

The audio-stream processing section 122 in Fig. 14 
includes the input portion 151, the filter control portion 
152, a PID filter 201, the base buffer 154, the first 
extension buffer 202, and the audio decoder 203. 

[0097] 

The filter control portion 152 controls the operation 
of the PID filter 201 based on the tables. Specifically, 
the filter control portion 152 remembers the types of 
streams that it can process and determines, based on the 
tables , the types of streams that it can process. The 
filter control portion 152 refers in the tables to the PIDs 
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of the streams that it can process and controls the PID 
filter 201 so as to select the TS packets having the PIDs 
associated with the streams that it can process . In the 
example of Fig. 14, the filter control portion 152 controls 
the PID filter 201 so as to supply the TS packets having PID 
= aO to the base buffer 154 and the TS packets having PID = 
al to the first extension buffer 202. Under the control of 
the filter control portion 152, the PID filter 201 extracts 
the TS packets. In other words, the PID filter 201 selects 
the TS packets having PID = aO and supplies the TS packets 
to the base buffer 154 and selects the TS packets having PID 
= al and supplies the TS packets to the first extension 
buffer 202. 
[0098] 

The base buffer 154 stores (buffers) the TS packets of 
the base audio stream, which are extracted by the PID filter 
201, and the first extension buffer 202 stores the TS 
packets of the first extension audio stream, which are 
extracted by the PID filter 201. The audio decoder 203 
acquires the TS packets stored in the base buffer 154 and 
the first extension buffer 202 and decodes the TS packets. 

[0099] 

In the example of Fig. 14, the audio-stream processing 
section 12 2 has the audio decoder 203 corresponding to the 
base audio stream and the first extension audio stream and 
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is incapable of decoding the second to n-th extension audio 
streams. In this case, under the control of the filter 
control portion 152 (determination by the filter control 
portion 152), the PID filter 201 extracts the TS packets 
with PID = aO and the TS packets with PID = al. In other 
words , when the TS packets of the second to n-th extension 
audio streams are supplied, the PID filter 201 does not 
select these packets. When the TS packets of the base audio 
stream having PID = aO are supplied, the PID filter 201 
selects the packets and supplies the packets to the base 
buffer 154 at a subsequent stage. When the TS packets of 
the first extension audio stream having PID = al are 
supplied, the PID filter 201 selects the packets and 
supplies the packets to the first extension buffer 202 at a 
subsequent stage . 
[0100] 

The base buffer 154 stores the TS packets of the base 
audio stream (PID = aO), which are selected and supplied by 
the PID filter 201. The base buffer 154 operates to allow 
synchronization of the TS packets at the receiving side and 
synchronization with the audio decoder 203. The first 
extension buffer 202 stores the TS packets of the first 
extension audio stream (PID = al), which are selected and 
supplied by the PID filter 201. The first extension buffer 
202 operates to allow synchronization of the TS packets at 
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the receiving side and synchronization with the audio 
decoder 203. Although one base buffer 154 and one extension 
buffer 202 are provided in the example of Fig. 14, two 
buffers may be provided in serial to serve as each buffer, 
including a buffer for allowing synchronization of the TS 
packets at the receiving side and a buffer for allowing 
synchronization with the audio decoder 203. The audio 
decoder 203 decodes the TS packets of the base audio stream 
and the TS packets of the first extension audio stream and 
outputs the decoded base audio stream and the decoded first 
extension audio stream. 
[0101] 

As described above, the receiver 42 (the audio-stream 
processing section 122 in Fig. 14) capable of decoding the 
base audio stream and the first extension audio stream can 
separate and play back only the base audio stream and the 
first extension audio stream. 
[0102] 

Next, the audio-stream processing section 122 having an 
audio decoder that can decode the base audio stream and the 
first and second extension audio streams is described with 
reference to Fig. 15. 
[0103] 

Fig. 15 is a diagram showing an example of the 
configuration of the audio-stream processing section 122 
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including an audio decoder that can decode the base audio 
stream and the first and second extension audio streams. In 
this diagram, portions corresponding to those in Figs. 11 
and 14 are referred to using the same reference numerals, 
and descriptions thereof are omitted to avoid repetition. 
[0104] 

The audio-stream processing section 122 in Fig. 15 
includes a first extension buffer 202-1 serving as the first 
extension buffer 202 in Fig. 14 and additionally includes a 
second extension buffer 202-2. An audio decoder 232 can 
decode not only the base audio stream and the first 
extension audio stream but also the second extension audio 
stream. A PID filter 231 can extract the base audio stream 
and the first and second extension audio streams. 

[0105] 

The audio- stream processing section 122 in Fig . 15 
includes the input portion 151, the filter control portion 
152, the PID filter 231, the base buffer 154, the first 
extension buffer 202-1, the second extension buffer 202-2, 
and the audio decoder 232. 

[0106] 

The filter control portion 152 remembers the types of 
streams that it can process and determines , based on the 
tables, the types of streams that it can process. The 
filter control portion 152 refers in the tables to the PIDs 
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of the streams that it can process and controls the PID 
filter 201 so as to select the TS packets having the PIDs 
associated with the streams that it can process. In the 
example of Fig. 15, the filter control portion 152 performs 
control so as to supply the TS packets having PID = aO to 
the base buffer 154, the TS packets having PID - al to the 
first extension buffer 202, and the TS packets having PID = 
a2 to the second extension buffer 202-2. Under the control 
of the filter control portion 152, the PID filter 231 
extracts the TS packets. In other words, the PID filter 231 
selects the TS packets having PID = aO and supplies the TS 
packets to the base buffer 154, selects the TS packets 
having PID = al and supplies the TS packets to the first 
extension buffer 202-1, and selects the TS packets having 
PID = a2 and supplies the TS packets to the second extension 
buffer 202-2. 
[0107] 

The second extension buffer 202-2 stores the TS packets 
of the second extension audio stream, which are selected by 
the PID filter 231. The audio decoder 232 acquires the TS 
packets stored in the base buffer 154 and the first and 
second extension buffers 202-1 and 202-2 and decodes the TS 
packets . 

[0108] 

In the example of Fig. 15, the audio-stream processing 
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section 122 has the audio decoder 232 corresponding to the 
base audio stream and the first and second extension audio 
streams and is incapable of decoding the third to n-th 
extension audio streams. In this case, under the control of 
the filter control portion 152, the PID filter 231 extracts 
(selects) the TS packets with PID = aO, the TS packets with 
PID = al, and the TS packets with PID = a2 . In other words, 
when the TS packets of the third to n-th extension audio 
streams are supplied, the PID filter 231 does not select 
these packets. When the TS packets of the base audio stream 
having PID = aO are supplied, the PID filter 231 selects the 
packets and supplies the packets to the base buffer 154 at a 
subsequent stage. When the TS packets of the first 
extension audio stream having PID = al are supplied, the PID 
filter 231 selects the packets and supplies the packets to 
the first extension buffer 202-1 at a subsequent stage. 
When the TS packets of the second extension audio stream 
having PID - a2 are supplied, the PID filter 231 selects the 
packets and supplies the packets to the second extension 
buffer 202-2 at a subsequent stage. 
[0109] 

The base buffer 154 stores the TS packets of the base 
audio stream (PID = aO), which are selected and supplied by 
the PID filter 231. The base buffer 154 operates to allow 
synchronization of the TS packets at the receiving side and 
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synchronization with the audio decoder 232. The first 
extension buffer 202-1 stores the TS packets of the first 
extension audio stream (PID = al) , which are selected and 
supplied by the PID filter 231. The first extension buffer 
202-1 operates to allow synchronization of the TS packets at 
the receiving side and synchronization with the audio 
decoder 232. The second extension buffer 202-2 stores the 
TS packets of the second extension audio stream (PID = a2) , 
which are selected and supplied by the PID filter 231. The 
first extension buffer 202-2 operates to allow 
synchronization of the TS packets at the receiving side and 
synchronization with the audio decoder 232. 
[0110] 

Although one base buffer 154, one extension buffer 202- 
1, and one extension buffer 202-2 are provided in the 
example of Fig. 15, two buffers may be provided in serial to 
serve as each buffer, including a buffer for allowing 
synchronization of the TS packets at the receiving side and 
a buffer for allowing synchronization with the audio decoder 
232. The audio decoder 232 decodes the TS packets of the 
base audio stream and the TS packets of the first and second 
extension audio streams and outputs the decoded base audio 
stream and the decoded first and second extension audio 
streams . 

[0111] 
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As described above, the receiver 42 (the audio-stream 
processing section 122 in Fig. 15) capable of decoding the 
base audio stream and the first and second extension audio 
streams can separate and play back the base audio stream and 
the first and second extension audio streams. 
[0112] 

Next, the audio-stream processing section 122 having an 
audio decoder that can decode the base audio stream and the 
first to n-th extension audio streams is described with 
reference to Fig. 16. 
[0113] 

Fig. 16 is a diagram showing an example of the 
configuration of the audio- stream processing section 122 
including an audio decoder that can decode the base audio 
stream and the first to n-th extension audio streams. In 
this diagram, portions corresponding to those in Fig. 15 are 
referred to using the same reference numerals, and 
descriptions thereof are omitted to avoid repetition. 

[0114] 

The audio-stream processing section 122 in Fig. 16 
includes, in addition to Fig. 15, third to n-th extension 
buffers 202-3 to 203-n. An audio decoder 262 can decode not 
only the base audio stream and the first and second 
extension audio streams but also the third to n-th extension 
audio streams. A PID filter 261 can extract the base audio 
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stream and even the first to n-th extension audio streams. 
[0115] 

The audio-stream processing section 122 in Fig. 16 
includes the input portion 151, the filter control portion 
152, the PID filter 261, the base buffer 154, the first to 
n-th extension buffers 202-1 to 202 -n, and the audio decoder 
262. 

[0116] 

The filter control portion 152 remembers that the base 
audio stream and the first to n-th extension audio streams 
are the types of streams that it can process. Based on the 
tables, the filter control portion 152 determines the types 
of streams that it can process. The filter control portion 
152 refers in the tables to the PIDs of the streams that it 
can process and controls the PID filter 261 so as to select 
the TS packets having the PIDs associated with the streams 
that it can process. In the example of Fig. 16, the filter 
control portion 152 controls the PID filter 261 so as to 
supply the TS packets having PID = aO to the base buffer 154, 
the TS packets having PID = al to the first extension buffer 
202-1, the TS packets having PID = a2 to the second 
extension buffer 202-2, and similarly the TS packets having 
PID = aN to the n-th extension buffer 202-n. Under the 
control of the filter control portion 152, the PID filter 
2 61 extracts the TS packets . In other words , the PID filter 
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261 selects the TS packets having PID = aO and supplies the 
TS packets to the base buffer 154; selects the TS packets 
having PID = al and supplies the TS packets to the first 
extension buffer 202-1; selects the TS packets having PID = 
a2 and supplies the TS packets to the second extension 
buffer 202-2; and similarly selects the TS packets having 
PID = aN and supplies the TS packets to the n-th extension 
buffer 202-n. 
[0117] 

The third to n-th extension buffers 202-3 to 202-n 
store the corresponding TS packets of the third to n-th 
extension audio streams, which are extracted by the PID 
filter 261. The audio decoder 262 acquires the TS packets 
stored in the base buffer 154 and the first to n-th 
extension buffers 202-1 to 202-n and decodes the TS packets. 

[0118] 

In the example of Fig. 16, the audio-stream processing 
section 122 has the audio decoder 262 corresponding to the 
base audio stream and the first to n-th extension audio 
streams, and, that is to say, is capable of decoding all the 
received extension audio streams up to the n-th level. In 
this case, under the control of the filter control portion 
152, the PID filter 261 extracts the TS packets with PIDs = 
aO to aN and supplies these TS packets to the corresponding 
buffers (the base buffer 154 and the first to n-th extension 
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buffers 202-1 to 202-n) . 
[0119] 

The base buffer 154 stores the TS packets of the base 
audio stream (PID = aO) , which are selected and supplied by 
the PID filter 261. The base buffer 154 operates to allow 
synchronization of the TS packets at the receiving side and 
synchronization with the audio decoder 262. The first 
extension buffer 202-1 stores the TS packets of the first 
extension audio stream (PID = al ) , which are selected and 
supplied by the PID filter 261. The first extension buffer 
202-1 operates to allow synchronization of the TS packets at 
the receiving side and synchronization with the audio 
decoder 262. The second extension buffer 202-2 stores the 
TS packets of the second extension audio stream (PID = a2), 
which are selected and supplied by the PID filter 261. The 
first extension buffer 202-2 operates to allow 
synchronization of the TS packets at the receiving side and 
synchronization with the audio decoder 262 . Similarly the 
n-th extension buffer 202-n stores the TS packets of the n- 
th extension audio stream (PID = aN) , which are selected and 
supplied by the PID filter 261. The first extension buffer 
2 02-n operates to allow synchronization of the TS packets at 
the receiving side and synchronization with the audio 
decoder 262. Although each of the base buffer 154 and the 
first to n-th extension buffers 202-1 to 202-n is provided 
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in the example of Fig. 16, two buffers may be provided in 
serial to serve as each buffer, including a buffer for 
allowing synchronization of the TS packets at the receiving 
side and a buffer for allowing synchronization with the 
audio decoder 262. The audio decoder 262 decodes the TS 
packets of the base audio stream and the TS packets of the 
first to n-th extension audio streams and outputs the 
decoded base audio stream and the decoded first to n-th 
extension audio streams . 
[0120] 

Next, an n-th audio stream processing process, which is 
an example of step S53 in Fig. 12, is described with 
reference to the flowchart of Fig . 17 . This process is a 
process executed by the audio-stream processing section 122 
in Fig. 16. Specifically, this process is a process 
executed by the audio-stream processing section 122 that can 
decode, in addition to the base audio stream, the first to 
n-th extension audio streams. 

[0121] 

In step S91, the input portion 151 receives input of 
the TS packets of the audio stream (the audio stream 
including the plural TS packets ) . The TS packets of the 
audio stream correspond to the audio stream including the TS 
packets of the tables, the TS packets of the base audio 
stream, and the TS packets of the first to n-th extension 
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audio streams, which are transmitted by the transmitter 41 
in step S19 of Fig. 6 described above. 
[0122] 

In step S92, the input portion 151 supplies the TS 
packets of the tables to the filter control portion 152. 
Specifically, since the TS packets of the audio stream 
include the TS packets of the tables, the TS packets of the 
base audio stream, and the TS packets of the first to n-th 
extension audio streams, the input portion 151 supplies, 
among these TS packets, the TS packets of the tables to the 
filter control portion 152. 

[0123] 

In step S93, the input portion 151 supplies the TS 
packets to which the PIDs are added to the PID filter 261. 
Specifically, in Fig. 5, the input portion 151 supplies the 
TS packets of the base audio stream and the first to n-th 
extension audio streams to which the PIDs are added to the 
PID filter 261. 

[0124] 

In step S94, the filter control portion 152 refers to 
the tables and determines the types of streams that the 
audio decoder 262 can process. Specifically, the filter 
control portion 152 remembers that the types of streams that 
it can process are the base audio stream and the first to n- 
th extension audio streams and determines, based on the 
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tables, the types of streams that it can process. That is, 
the filter control portion 152 determines, from the tables, 
the base audio stream and the first to n-th extension audios 
streams. In other words, the filter control portion 152 
refers to the tables and determines the types of streams 
that it can process and determines the PIDs associated with 
these streams as the types of streams that it can process. 
The filter control portion 152 controls the PID filter 153 
so as to select the TS packets having PID = aO associated 
with the base stream and the TS packets having PID - al to 
aN associated with the first to n-th extension audio streams. 
[0125] 

As described above, the filter control portion 152 
controls the PID filter 261 so as to supply the TS packets 
of the base audio stream and the TS packets of the first to 
n-th extension audio streams to the base buffer 154 and the 
first to n-th extension buffers 202-1 to 202-n, respectively. 
Since in the process in Fig. 17 the audio decoder 262 is 
capable of decoding the base audio stream and the first to 
n-th extension audio streams, the filter control portion 152 
controls the PID filter 261 so as to supply the TS packets 
having PID = aO, al, a2, . .., aN to the corresponding 
buffers . 

[0126] 

In step S95, based on the determination by the filter 



- 53 - 



control portion 152 (the types of streams determined to be 
processable) , the PID filter 261 selects a switch and 
supplies the corresponding TS packets to the buffers at a 
subsequent stage. Specifically, the PID filter 261 uses the 
switch to select the TS packets having PID = aO, namely, the 
TS packets of the base audio stream, and supplies the TS 
packets to the base buffer 154 at a subsequent stage. In 
addition, the PID filter 261 uses the switch to select the 
TS packets having PID = al to aN, namely, the TS packets of 
the first to n-th extension audio streams and supplies these 
TS packets to the corresponding first to n-th extension 
buffers 202-1 to 202-n at a subsequent stage. 
[0127] 

In step S96, the base buffer 154 and the first to n-th 
extension buffers 202-1 to 202-n store the supplied TS 
packets, respectively. 

[0128] 

In step S97, the base buffer 154 and the first to n-th 
extension buffers 202-1 to 202-n output the TS packets to 
the audio decoder 262 with predetermined timing. 

[0129] 

In step S98, the audio decoder 262 decodes the supplied 
TS packets of the base audio stream and, in step S99, 
outputs the decoded audio stream.. 

[0130] 
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As described above, the received stream includes the TS 
packets forming the base audio stream, the TS packets 
forming the first to n-th extension audio streams, and the 
TS packets of the tables in which information is written for 
associating the PIDs that respectively identify these TS 
packets with the base audio stream and the first to n-th 
extension audio streams. Since the PIDs that identify the 
types of audio streams are added to the TS packets forming 
the base audio stream and the TS packets forming the first 
to n-th extension audio streams, the audio-stream processing 
section 122 (the receiver 42) capable of decoding the base 
audio stream and the first to n-th extension audio streams, 
such as that shown in Fig. 17, can perform decoding. 

[0131] 

The receiver 4 2 capable of decoding the base audio 
stream and the first to m-th extension audio streams (m 
represents a natural number greater than or equal to one and 
less than or equal to n) can separate and play back the base 
audio stream and the first to m-th extension audio streams. 

[0132] 

Next, another example of the configuration of the 
receiver 42 including the audio-stream processing section 
122 capable of decoding the base audio stream and the first 
to n-th extension audio streams is described. Fig. 18 is a 
diagram showing an example of the configuration of the 
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audio-stream processing section 122 decoding the base audio 
stream and the first to n-th extension audio streams. 
[0133] 

The audio-stream processing section 122 in Fig. 18 is 
limited in the TS multiplexing . Specifically, it is 
necessary that, in a TS, encoding units to be synchronously 
played back be encoded in sequence in the order of the base 
audio stream and the first to n-th extension audio streams . 
That is, in a TS, encoding units need to be encoded in 
sequence in the order of BS(1), Extl(l), Ext2(l), ... 
ExtN(l), BS(2), Ext2(2), ... ExtN(2). In other words, the 
TS packets received by the receiving section 121 of the 
receiver 42 are input to the input portion 151 in the order 
of BS(1), Extl(l), Ext2(l), ... ExtN(l), BS(2), Ext2(2), ... 
ExtN(2), and the input portion 151 supplies the TS packets 
in the order of BS(1), Extl(l), Ext2(l), ... ExtN(l), BS(2), 
Ext2(2), ... ExtN(2) to a PID filter 301. 

[0134] 

The audio-stream processing section 122 in Fig. 18 
includes the input portion 151, the filter control portion 
152, the PID filter 301, a buffer 302, and an audio decoder 
303. 

[0135] 

The filter control portion 152 controls the PID filter 
301 so as to supply the TS packets having PID = aO to aN to 
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the buffer 302. Under the control of the filter control 
portion 152, the PID filter 301 extracts the TS packets. In 
other words, the PID filter 301 sequentially supplies the TS 
packets having PID = aO to aN to the buffer 302. Since the 
TS packets in the TS are encoded in sequence in the order of 
BS(1), Extl(l), Ext2(l), ... ExtN(l), BS(2), Ext2(2), ... 
ExtN(2), the PID filter 301 supplies the TS packets in the 
TS in the order of BS(1), Extl(l), Ext2(l), ... ExtN(l), 
BS(2), Ext2(2), ... ExtN(2) to the buffer 302. 
[0136] 

The buffer 302 stores the TS packets supplied in 
sequence in the order of BS(1), Extl(l), Ext2(l), ... 
ExtN(l), BS(2), Ext2(2), ... ExtN(2). The audio decoder 303 
acquires the TS packets stored in the buffer 302 and decodes 
the TS packets . 

[0137] 

In the example of Eig. 18, the audio-stream processing 
section 122 has the audio decoder 303 corresponding to the 
base audio stream and the first to n-th extension audio 
streams and thus is capable of decoding all of the received 
base audio stream and the first to n-th extension audio 
streams . In this case , under the control of the filter 
control portion 152, the PID filter 301 extracts the TS 
packets with PID = aO to aN and supplies these TS packets to 
the buffer 302. 
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[0138] 

The buffer 302 stores the TS packets of the audio 
stream (PID = aO to aN) , which are sequentially selected and 
supplied by the PID filter 301. The buffer 302 operates to 
allow synchronization of the TS packets at the receiving 
side and synchronization with the audio decoder 303. 
Although one buffer 302 is provided in the example of Fig. 
18, two buffers may be provided in serial to serve as the 
buffer 302, including a buffer for allowing synchronization 
of the TS packets at the receiving side and a buffer for 
allowing synchronization with the audio decoder 303. The 
audio decoder 303 sequentially decodes the TS packets of the 
base audio stream and the TS packets of the first to n-th 
extension audio streams and outputs the decoded base audio 
stream and the decoded first to n-th extension audio streams. 
[0139] 

According to Fig. 18, the number of buffers is reduced 
compared with that in Fig. 16, thereby achieving lower cost. 
Although the number of buffers in Fig. 16 is greater than 
that in Fig. 18, it is advantageous in that the order of 
encoding TS packets is not limited. 
[0140] 

According to the embodiment, in the case where a 
multiplexed stream including the base audio stream and 
extension audio streams at a plurality of levels is played 
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back, there is provided a multiplexed stream encoding and 
decoding method for the receiver 42 capable of decoding only 
the base audio stream (e.g., the receiver 42 having the 
audio-stream processing section 122 in Fig. 11) that 
separates and plays back only the base audio stream; and for 
the playback apparatus 42 capable of playing back the 
extension audio streams up to a predetermined n-th level 
(e.g., the receiver 42 having the audio-stream processing 
section 122 in Fig. 16) that separates and plays back the 
base audio stream and the extension audio streams up to the 
n-th level. 
[0141] 

That is, a stream including the base audio stream and 
extension streams at a plurality of levels can be encoded so 
as to be decodable in accordance with the processing 
capacity of the receiving side. In other words, when a 
stream including the base audio stream and extension streams 
at a plurality of levels is transmitted, an apparatus at the 
receiving side can determine the types of streams that it 
can process and can decode and play back only the streams 
that it can process. 

[0142] 

Since it is only necessary to add the PID to each TS 
packet and to add the tables, this method is easily 
applicable to a transport stream. 
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[0143] 

Since only one type of stream needs to be included in 
one TS packet having 188 bytes, which is relatively a small 
amount of data, that is, the base audio stream and the 
extension audio stream ( s ) need not be included in one TS 
packet, encoding with high encoding efficiency can be 
performed. 

[0144] 

Because the stream structure is not defined by the 
format, even when an extension is added to an audio stream, 
that is, even when n is increased, both encoding and 
decoding in accordance therewith can be performed. That is, 
even an apparatus capable of decoding only the base audio 
stream can decode an audio stream to which extension is 
added. 

[0145] 

The present invention is not limited to the transmitter 

41 having an encoder and is applicable to all the 
information processing apparatuses that perform encoding . 
Also, the present invention is not limited to the receiver 

42 having a decoder and is applicable to all the information 
processing apparatuses that perform decoding . 

[0146] 

In the above examples, the cases in which the present 
invention is applied to encoding and decoding of an audio 
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stream have been described . However, the present invention 
is not limited thereto but can be applied to encoding and 
decoding of a video stream. In other words, the present 
invention can be applied to a stream such as an audio stream 
or a video stream. 
[0147] 

The above-described consecutive processing can be 
executed by hardware or can be executed by software. In 
this case, the above-described processing is executed by a 
personal computer 500 shown in Fig. 20. 

[0148] 

In Fig. 20, a CPU 501 executes various types of 
processing in accordance with programs stored in a ROM 502 
or programs loaded from a storage section 508 to a RAM 503. 
The RAM 503 stores the data required for the CPU 501 to 
execute various types of processing. 

[0149] 

The CPU 501, the ROM 502, and the RAM 503 are connected 
to one another via an internal bus 504. The internal bus 
504 also connects to an input/output interface 505. 

[0150] 

The input/output interface 505 connects to an input 
section 506 including a keyboard and a mouse; an output 
section 507 including a display formed by a CRT, an LCD, or 
the like, and a speaker; a storage section 508 including a 
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hard disk; and a communication section 509 including a modem 
and a terminal adapter. The communication section 509 
performs communicating processing via various types of 
networks including a telephone line and a CATV. 
[0151] 

The input/output interface 505 connects to a drive 510, 
if necessary, and a removable medium 521, such as a magnetic 
disk, an optical disc, a magneto-optical disc, or a 
semiconductor memory, is loaded into the drive 510. A 
computer program read therefrom is installed in the storage 
section 508, if necessary. 

[0152] 

When the consecutive processing is executed by software, 
programs forming the software are installed from a network 
or a recording medium. 
[0153] 

As shown in Fig. 20, the recording medium includes a 
package medium which includes the program-recorded removable 
medium 521 distributed for providing a user with a program, 
but also a hard disk which includes the program-recorded ROM 
502 and the program-recorded storage section 508 provided to 
the user in a form built into an apparatus. 

[0154] 

In this specification, steps constituting a computer 
program include, needless to say, processing steps which are 
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performed in a time-series manner in the order given, and 
include processing steps which are executed in parallel or 
separately even if they are not always executed in a time- 
series manner . 
[0155] 

In addition, in this specification, the system means 
the entirety of an apparatus including a plurality of 
devices . 

[Brief Description of the Drawings] 
[0156] 

[Fig. 1] Fig. 1 is an illustration of the structure of a 
program stream in the DVD video format. 

[Fig. 2] Fig. 2 is a diagram showing an example of the 
configuration of the entirety of a transmitting/receiving 
system to which the present invention is applied. 

[Fig. 3] Fig. 3 is a block diagram showing an example of 
the configuration of a transmitter in Fig. 2. 

[Fig. 4] Fig. 4 is an illustration of the structure of a 
TS in which a base audio stream and extension audio streams 
at a plurality of levels are multiplexed. 

[Fig. 5] Fig. 5 is an illustration of the structure of 
the TS in which the base audio stream and the extension 
audio streams at a plurality of levels are multiplexed. 

[Fig. 6] Fig. 6 is a flowchart illustrating a TS packet 
transmitting process executed by the transmitter in Fig. 3. 
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[Fig. 7] Fig. 7 is an illustration of a table. 

[Fig. 8] Fig. 8 is an illustration of a table. 

[Fig. 9] Fig. 9 is a flowchart illustrating a TS packet 
generating process . 

[Fig. 10] Fig. 10 is a block diagram showing an example 
of the configuration of a receiver in Fig. 2. 

[Fig. 11] Fig. 11 is an illustration of an example of the 
configuration of an audio-stream processing section that can 
decode only the base audio stream. 

[Fig. 12] Fig. 12 is a flowchart showing a TS packet 
receiving process executed by the receiver in Fig. 10 . 

[Fig. 13] Fig. 13 is a flowchart showing an audio stream 
processing process executed by the audio-stream processing 
section in Fig. 11. 

[Fig. 14] Fig. 14 is an illustration of an example of the 
configuration of an audio- stream processing section that can 
decode the base audio stream and the first extension audio 
stream. 

[Fig. 15] Fig. 15 is an illustration of an example of the 
configuration of an audio-stream processing section that can 
decode the base audio stream and the first and second 
extension audio streams . 

[Fig. 16] Fig. 16 is an illustration of an example of the 
configuration of an audio-stream processing section that can 
decode the base audio stream and the first to n-th extension 
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audio streams. 

[Fig. 17] Fig. 17 is a flowchart showing an audio stream 
processing process executed by the audio-stream processing 
section in Fig. 16. 

[Fig. 18] Fig. 18 is an illustration of an example of the 
configuration of the audio-stream processing section that 
can decode the base audio stream and the first to n-th 
extension audio streams . 

[Fig. 19] Fig. 19 is an illustration of the order of TS 
packets . 

[Fig. 20] Fig. 20 is a diagram showing a personal 
computer . 

[Reference Numerals ] 
[0157] 

40 transmitting/receiving system, 41 transmitter, 42 
receiver, 71 input section, 72 audio encoder, 73 base 
buffer, 74-1 to 74-n extension buffers, 76 TS 
packetizing section, 77 transmitting section, 90 to 92, 
and 93-1 to 93-n TS packet, 121 receiving section, 122 
audio-stream processing section, 123 output section, 151 
input portion, 152 filter control portion, 153 PID 
filter, 154 base buffer, 155 audio decoder, 201 PID 
filter, 202 extension buffer, 203 audio decoder, 231 
PID filter, 232 audio decoder, 261 PID filter, 262 
audio decoder, 301 PID filter, 302 buffer, 303 audio 
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Ext2_PID a2 

ExtN_PlD aN 

} 

} 
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[Fig. 1] 

1: PROGRAM STREAM 

11: VIDEO PACK 

12: MPEG2 AUDIO PACK 

13-1 ... 13-J: PACK 

21: PACK HEADER 

22: PES PACKET HEADER 

23: MPEG-1 AUDIO DATA (Base) 

24: PES PACKET HEADER 

25: MPEG-2 AUDIO DATA (Extension) 

[Fig. 2] 

40: TRANSMITTING/RECEIVING SYSTEM 
41 : TRANSMITTER 
42: RECEIVER 
[Fig. 3] 

300: AUDIO STREAM 
41: TRANSMITTER 
71: INPUT SECTION 
72: AUDIO ENCODER 
73: BASE BUFFER 

74-1, 74-2 ... 74-N: EXTENSION BUFFER 

75: EXTRACTION-INFORMATION ADDING SECTION 

76: TS PACKETI Z ING SECTION 

77: TRANSMITTING SECTION 

[Fig. 4] 
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81: BASE AUDIO STREAM (BS) 

82-1: FIRST EXTENSION AUDIO STREAM (Extl) 
82-2: SECOND EXTENSION AUDIO STREAM (Ext2) 
82-n: N-TH EXTENSION AUDIO STREAM (ExtN) 
[Fig. 5] 

500: TRANSPORT STREAM (TS) 
501: TS PACKET 
90: TABLE (PAT) 
91 : TABLE (PMT) 
[Fig. 6] 

600: START OF TS PACKET TRANSMITTING PROCESS 

Sll: INPUT SECTION RECEIVES INPUT OF AUDIO STREAM 

S12: INPUT SECTION OUTPUTS AUDIO STREAM TO AUDIO ENCODER 

S13: AUDIO ENCODER ENCODES AUDIO STREAM SEPARATELY FOR EACH 

LEVEL 

S14: AUDIO ENCODER OUTPUTS ENCODED AUDIO STREAMS SEPARATELY 
FOR EACH LEVEL 

S15: BASE BUFFER AND FIRST TO N-TH EXTENSION BUFFERS STORE 
ENCODED AUDIO STREAMS 

S16: BASE BUFFER AND FIRST TO N-TH EXTENSION BUFFERS 
RESPECTIVELY OUTPUT ENCODED AUDIO STREAMS WITH PREDETERMINED 
TIMING 

S17: EXTRACTION- INFORMATION ADDING SECTION GENERATES TABLES 
AND SUPPLIES TABLES TO TS PACKETIZING SECTION 
S18: TS PACKET GENERATING PROCESS 
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S19: TRANSMITTING SECTION TRANSMITS TS PACKETS 
601: END 
[Fig. 9] 

900: START OF TS PACKET GENERATING PROCESS 
S31: ACQUIRE TABLES 

S32: PACKETIZE TABLES INTO TS PACKETS AND OUTPUT TS PACKETS 
TO TRANSMITTING SECTION 

S33: ADDS PIDS TO AUDIO STREAMS FROM BASE BUFFER AND FIRST 
TO N-TH EXTENSION BUFFERS BASED ON TABLES 

S34: GENERATE TS PACKETS BASED ON AUDIO STREAMS FROM BASE 
BUFFER AND FIRST TO N-TH EXTENSION BUFFERS 

S35: OUTPUT GENERATED TS PACKETS TO TRANSMITTING SECTION 

901: RETURN 

[Fig. 10] 

42 : RECEIVER 

121: RECEIVING SECTION 

122: AUDIO-STREAM PROCESSING SECTION 
123: OUTPUT SECTION 
[Fig. 11] 

122: AUDIO-STREAM PROCESSING SECTION 

151: INPUT PORTION 

152: FILTER CONTROL PORTION 

153: PID FILTER 

154: BASE BUFFER 

155: AUDIO DECODER 
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1100: DECODED BASE AUDIO STREAM 
[Fig. 12] 

1200: START OF TS PACKET GENERATING PROCESS 
S51: RECEIVING SECTION RECEIVES TS PACKETS 

S52: RECEIVING SECTION EXTRACTS TS PACKETS OF AUDIO STREAM 
AND SUPPLIES TS PACKETS TO AUDIO-STREAM PROCESSING SECTION 
S53: AUDIO STREAM PROCESSING PROCESS 
S54: OUTPUT SECTION OUTPUTS AUDIO STREAM 
1201: END 
[Fig. 13] 

1300: START OF BASE AUDIO STREAM PROCESSING PROCESS 

S71: INPUT PORTION RECEIVES INPUT OF TS PACKETS OF AUDIO 

STREAM 

S72: INPUT PORTION SUPPLIES TS PACKETS OF TABLES TO FILTER 
CONTROL PORTION 

S73: INPUT PORTION SUPPLIES TS PACKETS TO WHICH PIDS ARE 
ADDED TO PID FILTER 

S74: FILTER CONTROL PORTION REFERS TO TABLES AND DETERMINES 

TYPE OF STREAM THAT AUDIO DECODER CAN PROCESS 

S75: PID FILTER SELECTS CORRESPONDING TS PACKETS AND 

SUPPLIES TS PACKETS TO BASE BUFFER 

S7 6: BASE BUFFER STORES TS PACKETS 

S77: BASE BUFFER OUTPUTS TS PACKETS TO AUDIO DECODER WITH 
PREDETERMINED TIMING 

S78: AUDIO DECODER DECODES TS PACKETS 
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S7 9: AUDIO DECODER OUTPUTS DECODED AUDIO STREAM 
1301: RETURN 
[Fig. 14] 

122: AUDIO-STREAM PROCESSING SECTION 

151: INPUT PORTION 

152: FILTER CONTROL PORTION 

201: PID FILTER 

154: BASE BUFFER 

202: EXTENSION BUFFER 

203: AUDIO DECODER 

1400: DECODED BASE AUDIO STREAM AND FIRST EXTENSION AUDIO 

STREAM 

[Fig. 15] 

122: AUDIO-STREAM PROCESSING SECTION 

151: INPUT PORTION 

152: FILTER CONTROL PORTION 

231: PID FILTER 

154: BASE BUFFER 

202-1, 202-2: EXTENSION BUFFER 
232: AUDIO DECODER 

1500: DECODED BASE AUDIO STREAM AND FIRST AND SECOND 
EXTENSION AUDIO STREAMS 
[Fig. 16] 

122: AUDIO-STREAM PROCESSING SECTION 
151: INPUT PORTION 
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152: FILTER CONTROL PORTION 
261: PID FILTER 
154: BASE BUFFER 

202-1, 202-2 ... 202-N: EXTENSION BUFFER 
262: AUDIO DECODER 

1600: DECODED BASE AUDIO STREAM AND FIRST TO N-TH EXTENSION 
AUDIO STREAMS 
[Fig. 17] 

1700: START OF N-TH AUDIO STREAM PROCESSING PROCESS 

S91: INPUT PORTION RECEIVES INPUT OF TS PACKETS OF AUDIO 

STREAM 

S92: INPUT PORTION SUPPLIES TS PACKETS OF TABLES TO FILTER 
CONTROL PORTION 

S93: INPUT PORTION SUPPLIES TS PACKETS TO WHICH PIDS ARE 
ADDED TO PID FILTER 

S94: FILTER CONTROL PORTION REFERS TO TABLES AND DETERMINES 
TYPES OF STREAMS THAT AUDIO STREAM CAN PROCESS 
S95: PID FILTER SELECTS SWITCH BASED ON DETERMINATION BY 
FILTER CONTROL PORTION 

S96: BASE BUFFER AND FIRST TO N-TH EXTENSION BUFFERS STORE 
CORRESPONDING TS PACKETS 

S97: BASE BUFFER AND FIRST TO N-TH EXTENSION BUFFERS OUTPUT 
TS PACKETS TO AUDIO DECODER WITH PREDETERMINED TIMING 
S98: AUDIO DECODER DECODES TS PACKETS 
S99: AUDIO DECODER OUTPUTS DECODED AUDIO STREAM 
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1701: RETURN 
[Fig. 18] 

122: AUDIO-STREAM PROCESSING SECTION 

151: INPUT PORTION 

152: FILTER CONTROL PORTION 

301: PID FILTER 

302: BUFFER 

303: AUDIO DECODER 

1800: DECODED BASE AUDIO STREAM AND FIRST TO N-TH EXTENSION 
AUDIO STREAMS 
[Fig. 20] 

500: PERSONAL COMPUTER 

504: INTERNAL BUS 

505: INPUT/OUTPUT INTERFACE 

506: INPUT SECTION 

507: OUTPUT SECTION 

508: STORAGE SECTION 

509: COMMUNICATION SECTION 

510: DRIVE 

521: REMOVABLE MEDIUM 
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[Name of Document] ABSTRACT 
[Abstract] 

[Object] To process a stream including a base stream and 
extension streams at a plurality of levels. 

[Solving Means] A transmitter encodes an audio stream 
into a base stream BS and first to n-th extension streams 
Extl to ExtN. The transmitter refers to tables 90 and 91, 
generates TS packets 92 and 93-1 to 93-n having PIDs for the 
corresponding types of streams and TS packets 90 and 91 of 
the tables, and transmits the generated TS packets to a 
receiver. The receiver refers to the TS packets 90 and 91 
of the tables and determines the type(s) of stream(s) that 
it can process. The receiver selects the TS packets having 
the PID(s) associated with the stream(s) that it can process 
and decodes the TS packets. The present invention is 
applicable to a transmitter and a receiver that perform 
encoding and decoding. 
[Selected Figure] Fig. 5 
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